Interactive recipe preparation using instructive device with integrated actuators to provide tactile feedback

ABSTRACT

A way of delivering recipe preparation instruction to disabled individuals is provided using an interactive cooking preparation device. The device retrieves an instruction delivery preference that corresponds to a user with a disability, such as a hearing or sight disability. The user then selects a recipe from list of recipes. Preparation steps that correspond to the selected recipe are retrieved from a data store, such as a database. The retrieved preparation steps are provided to the user using the interactive cooking preparation device that provides the preparation steps in an alternative delivery mode based on the user&#39;s delivery preference.

BACKGROUND OF THE INVENTION

1. Technical Field

The present invention relates to an approach for using a computerizedinteractive instruction device that provides interactive recipepreparation instructions to disabled individuals.

2. Description of the Related Art

Various sources are available to assist a user in preparing meals andfollowing a recipe. These sources include books, television programs,videos, and Internet sources. These sources provide cooking guidance andteaching assisting users with a variety of skill levels. These sourcesoften provide step-by-step instructions.

A challenge of these sources is that they are not interactive and theyare not generally tailored to the needs of disabled individuals, such asindividuals with sight or hearing impairments. One approach to thischallenge is a live cooking instructor with whom the user can interact.However, cooking instruction is quite costly, especially personalinstruction, and is generally provided at a location outside the user'shome, such as at a cooking school or restaurant. Implements andingredients found at cooking schools and restaurants is often quitedifferent from the implements and ingredients found at the user's home.These differences make it challenging to apply the instruction providedat the cooking school or restaurant with recipe preparation at theuser's normal location, such as the user's home kitchen. Moreover,instructors at these types of cooking schools are often ill-equipped andnot trained for the specific needs of disabled individuals.

Another challenge is that traditional sources do not provide additionalcomputerized intelligence to assist the user during meal preparation. Inaddition, traditional sources do not interface with the user'sappliances in order to further assist the user. Finally, traditionalsources do not monitor the user's performance nor do they alert the userwhen the user is not following the recipe correctly.

SUMMARY

It has been discovered that the aforementioned challenges are resolvedby using an interactive cooking preparation device. The interactivecooking preparation device retrieves an instruction delivery preferencethat corresponds to a user with a disability, such as a hearing or sightdisability. The user then selects a recipe from list of recipes.Preparation steps that correspond to the selected recipe are retrievedfrom a data store, such as a database. The retrieved preparation stepsare provided to the user using the interactive cooking preparationdevice that provides the preparation steps in an alternative deliverymode based on the user's delivery preference. For example, if the useris hearing disabled, the interactive cooking preparation device can beset to display subtitles (e.g., closed captioning) as well as visuallyhighlight surface areas on the interactive cooking preparation device.If the user is visually impaired, the interactive cooking preparationdevice can be set to provide audible instructions, Braille instructions,or tactile cues (e.g., vibration) to the user.

The foregoing is a summary and thus contains, by necessity,simplifications, generalizations, and omissions of detail; consequently,those skilled in the art will appreciate that the summary isillustrative only and is not intended to be in any way limiting. Otheraspects, inventive features, and advantages of the present invention, asdefined solely by the claims, will become apparent in the non-limitingdetailed description set forth below.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention may be better understood, and its numerousobjects, features, and advantages made apparent to those skilled in theart by referencing the accompanying drawings, wherein:

FIG. 1 is a block diagram of a data processing system in which themethods described herein can be implemented;

FIG. 2 provides an extension of the information handling systemenvironment shown in FIG. 1 to illustrate that the methods describedherein can be performed on a wide variety of information handlingsystems which operate in a networked environment;

FIG. 3 is a diagram of the components used in an interactive cookingpreparation device that is an extension of the information handlingsystem shown in FIG. 1;

FIG. 4 is a flowchart showing steps taken to setup and modify a userprofile used in the interactive cooking preparation device;

FIG. 5 is a flowchart showing steps taken to select a recipe using theinteractive cooking preparation device;

FIG. 6 is a flowchart showing steps taken to interactively prepare arecipe using the interactive cooking preparation device;

FIG. 7 is a flowchart showing steps taken to setup ingredients usedduring recipe preparation using the interactive cooking preparationdevice;

FIG. 8 is a flowchart showing steps taken to setup implements usedduring recipe preparation using the interactive cooking preparationdevice;

FIG. 9 is a flowchart of the processing performed to perform the stepsset forth in a recipe;

FIG. 10 is a flowchart of the steps taken to monitor a user's actionswhen following steps in a recipe;

FIG. 11 is a flowchart of the steps taken to deliver instructionmaterial to the user of the interactive cooking preparation device;

FIG. 12 is a diagram of a flexible horizontal display screen withactuators underneath to provide tactile feedback to a user of theinteractive cooking preparation device;

FIG. 13 is a flowchart of the steps taken to provide enhancedinstructional delivery to users with a visual impairment;

FIG. 14 is a flowchart of the steps taken to provide enhancedinstructional delivery to users with a hearing impairment;

FIG. 15 is a flowchart of the steps taken to identify items usingvarious scanners connected to the interactive cooking preparationdevice;

FIG. 16 is a flowchart of the steps taken to provide standard deliveryof instructional materials based on user preferences;

FIG. 17 is a flowchart showing the steps taken to retrieve implementusage instructions and provide the instructions to the user of theinteractive cooking preparation device;

FIG. 18 is a flowchart showing the steps taken to identify and suggestadditional recipes that complement a recipe selected by the user of theinteractive cooking preparation device;

DETAILED DESCRIPTION

The following is intended to provide a detailed description of anexample of the invention and should not be taken to be limiting of theinvention itself. Rather, any number of variations may fall within thescope of the invention, which is defined in the claims following thedescription.

Certain specific details are set forth in the following description andfigures to provide a thorough understanding of various embodiments ofthe invention. Certain well-known details often associated withcomputing and software technology are not set forth in the followingdisclosure, however, to avoid unnecessarily obscuring the variousembodiments of the invention. Further, those of ordinary skill in therelevant art will understand that they can practice other embodiments ofthe invention without one or more of the details described below.Finally, while various methods are described with reference to steps andsequences in the following disclosure, the description as such is forproviding a clear implementation of embodiments of the invention, andthe steps and sequences of steps should not be taken as required topractice this invention. Instead, the following is intended to provide adetailed description of an example of the invention and should not betaken to be limiting of the invention itself. Rather, any number ofvariations may fall within the scope of the invention, which is definedby the claims that follow the description.

The following detailed description will generally follow the summary ofthe invention, as set forth above, further explaining and expanding thedefinitions of the various aspects and embodiments of the invention asnecessary. To this end, this detailed description first sets forth acomputing environment in FIG. 1 that is suitable to implement thesoftware and/or hardware techniques associated with the invention. Anetworked environment is illustrated in FIG. 2 as an extension of thebasic computing environment, to emphasize that modern computingtechniques can be performed across multiple discrete devices.

FIG. 1 illustrates information handling system 100 which is a simplifiedexample of a computer system capable of performing the computingoperations described herein. Information handling system 100 includesone or more processors 110 which are coupled to processor interface bus112. Processor interface bus 112 connects processors 110 to Northbridge115, which is also known as the Memory Controller Hub (MCH). Northbridge115 is connected to system memory 120 and provides a means forprocessor(s) 110 to access the system memory. Graphics controller 125 isalso connected to Northbridge 115. In one embodiment, PCI Express bus118 is used to connect Northbridge 115 to graphics controller 125.Graphics controller 125 is connected to display device 130, such as acomputer monitor.

Northbridge 115 and Southbridge 135 are connected to each other usingbus 119. In one embodiment, the bus is a Direct Media Interface (DMI)bus that transfers data at high speeds in each direction betweenNorthbridge 115 and Southbridge 135. In another embodiment, a PeripheralComponent Interconnect (PCI) bus is used to connect the Northbridge andthe Southbridge. Southbridge 135, also known as the I/O Controller Hub(ICH) is a chip that generally implements capabilities that operate atslower speeds than the capabilities provided by the Northbridge.Southbridge 135 typically provides various busses used to connectvarious components. These busses can include PCI and PCI Express busses,an ISA bus, a System Management Bus (SMBus or SMB), a Low Pin Count(LPC) bus. The LPC bus is often used to connect low-bandwidth devices,such as boot ROM 196 and “legacy” I/O devices (using a “super I/O”chip). The “legacy” I/O devices (198) can include serial and parallelports, keyboard, mouse, floppy disk controller. The LPC bus is also usedto connect Southbridge 135 to Trusted Platform Module (TPM) 195. Othercomponents often included in Southbridge 135 include a Direct MemoryAccess (DMA) controller, a Programmable Interrupt Controller (PIC), astorage device controller, which connects Southbridge 135 to nonvolatilestorage device 300 such as a hybrid hard disk drive, using bus 184.

ExpressCard 155 is a slot used to connect hot-pluggable devices to theinformation handling system. ExpressCard 155 supports both PCI Expressand USB connectivity as it is connected to Southbridge 135 using boththe Universal Serial Bus (USB) the PCI Express bus. Southbridge 135includes USB Controller 140 that provides USB connectivity to devicesthat connect to the USB. These devices include webcam (camera) 150,infrared (IR) receiver 148, Bluetooth device 146 which provides forwireless personal area networks (PANs), keyboard and trackpad 144, andother miscellaneous USB connected devices 142, such as a mouse,removable nonvolatile storage device 145, modems, network cards, ISDNconnectors, fax, printers, USB hubs, and many other types of USBconnected devices. While removable nonvolatile storage device 145 isshown as a USB-connected device, removable nonvolatile storage device145 could be connected using a different interface, such as a Firewireinterface, etc. Removable storage device 145 can also be a hybrid diskdrive, such as hybrid disk drive 300 shown in FIGS. 3-6.

Wireless Local Area Network (LAN) device 175 is connected to Southbridge135 via the PCI or PCI Express bus 172. LAN device 175 typicallyimplements one of the IEEE 802.11 standards of over-the-air modulationtechniques that all use the same protocol to wireless communicatebetween information handling system 100 and another computer system ordevice. Optical storage device 190 is connected to Southbridge 135 usingSerial ATA (SATA) bus 188. Serial ATA adapters and devices communicateover a high-speed serial link. The Serial ATA bus is also used toconnect Southbridge 135 to other forms of storage devices, such as harddisk drives. Audio circuitry 160, such as a sound card, is connected toSouthbridge 135 via bus 158. Audio circuitry 160 is used to providefunctionality such as audio line-in and optical digital audio in port162, optical digital output and headphone jack 164, internal speakers166, and internal microphone 168. Ethernet controller 170 is connectedto Southbridge 135 using a bus, such as the PCI or PCI Express bus.Ethernet controller 170 is used to connect information handling system100 with a computer network, such as a Local Area Network (LAN), theInternet, and other public and private computer networks.

While FIG. 1 shows one information handling system, an informationhandling system may take many forms. For example, an informationhandling system may take the form of a desktop, server, portable,laptop, notebook, or other form factor computer or data processingsystem. In addition, an information handling system may take other formfactors such as a personal digital assistant (PDA), a gaming device, ATMmachine, a portable telephone device, a communication device or otherdevices that include a processor and memory.

The Trusted Platform Module (TPM 195) shown in FIG. 1 and describedherein to provide security functions is but one example of a hardwaresecurity module (HSM). Therefore, the TPM described and claimed hereinincludes any type of HSM including, but not limited to, hardwaresecurity devices that conform to the Trusted Computing Groups (TCG)standard, and entitled “Trusted Platform Module (TPM) SpecificationVersion 1.2.” The TPM is a hardware security subsystem that may beincorporated into any number of information handling systems, such asthose outlined in FIG. 2.

FIG. 2 provides an extension of the information handling systemenvironment shown in FIG. 1 to illustrate that the methods describedherein can be performed on a wide variety of information handlingsystems which operate in a networked environment. Types of informationhandling systems range from small handheld devices, such as handheldcomputer/mobile telephone 210 to large mainframe systems, such asmainframe computer 270. Examples of handheld computer 210 includepersonal digital assistants (PDAs), personal entertainment devices, suchas MP3 players, portable televisions, and compact disc players. Otherexamples of information handling systems include pen, or tablet,computer 220, laptop, or notebook, computer 230, workstation 240,personal computer system 250, and server 260. Other types of informationhandling systems that are not individually shown in FIG. 2 arerepresented by information handling system 280. As shown, the variousinformation handling systems can be networked together using computernetwork 200. Types of computer network that can be used to interconnectthe various information handling systems include Local Area Networks(LANs), Wireless Local Area Networks (WLANs), the Internet, the PublicSwitched Telephone Network (PSTN), other wireless networks, and anyother network topology that can be used to interconnect the informationhandling systems. Many of the information handling system includenonvolatile data stores, such as hard drives and/or nonvolatile memory.Some of the information handling systems shown in FIG. 2 are depictedwith separate nonvolatile data stores (server 260 is shown withnonvolatile data store 265, mainframe computer 270 is shown withnonvolatile data store 275, and information handling system 280 is shownwith nonvolatile data store 285). The nonvolatile data store can be acomponent that is external to the various information handling systemsor can be internal to one of the information handling systems. Inaddition, removable nonvolatile storage device 145 can be shared amongsttwo or more information handling systems using various techniques, suchas connecting the removable nonvolatile storage device 145 to a USB portor other connector of the information handling systems.

FIG. 3 is a diagram of the components used in an interactive cookingpreparation device that is an extension of the information handlingsystem shown in FIG. 1. Information handling system 100, which is aninteractive cooking preparation device, includes integrated interactivevideo display 310 which is a touch screen display. In one embodiment,display 310 is mounted horizontally so that items 390 can be placed uponthe display. In a further embodiment, the horizontal display is aflexible surface (such as a Flexible Organic Light-Emitting Diode(Flexible OLED or FOLED) display panel). Device sensors 315 are includedin the interactive cooking preparation device. In one embodiment, devicesensors 315 are barcode sensors, in another embodiment the sensors areRF tag or other tag readers, and in another embodiment the sensors are acombination of barcode and tag reading sensors. Sensors 315 are used tosense various items 390 placed on horizontal surface 310. These itemsmay include various implements (e.g., knives, spoons, etc.), ingredientsused to prepare recipes, kitchen appliances (e.g., mixers, etc.) and anyother item used to prepare recipes. Items are affixed with anidentifier, such as a barcode or a tag, that can be sensed by sensors315 when the item is placed in proximity to one or more of the sensors.One or more integrated scales 320 are also included in the interactivecooking preparation device. Integrated scales 320 are used to weighitems 390 placed on the surface of the interactive cooking preparationdevice. In one embodiment, the interactive cooking preparation deviceincludes microphone 325 to allow the user to communicate with theinteractive cooking preparation device using voice responses andcommands. As noted, one of user actions 395 include voice input that isreceived by microphone 325. One or more speakers 330 are included in theinteractive cooking preparation device in order to provide the user withaudible output, such as audible output used in recipe preparation. Inaddition, audible output from speakers 330 can be combined withmultimedia content where a video is displayed on horizontal displayscreen 310 coupled with audio that is played through speakers 330.

Video input devices 335 are also included in the interactive cookingpreparation device. Video input devices 335 can capture images of items390 that are in proximity of the interactive cooking preparation device.In this manner, video input devices 335 can be used as additional sensordevices, especially when an item does not have an identifier and is,instead, identified based on the items shape or appearance using one ormore video input devices. In addition, video input devices 335 captureuser actions 395, such as the user performing a preparation step, makingvarious gestures, and the like. Tactile output mechanism 340 is amechanism used to provide tactile feedback to the user. In oneembodiment, actuators are located under flexible video display panel 310and, when actuated, provide a sensation on the horizontal surface of theinteractive cooking preparation device that can be felt by the user.

Various data stores (e.g., databases, flat files, etc.) are stored innonvolatile storage devices accessible from the interactive cookingpreparation device 100. As shown, these data stores include recipes datastore 350, procedures data store 360, ingredients data store 370,ingredients inventory data store 375, implements data store 380, andimplements inventory data store 385. Recipes data store 350 includes avariety recipes with information related to each recipe. Recipes datastore 350 includes the following information regarding recipes:

-   -   Name of the recipe;    -   Reference Source where recipe is found (e.g., name of author,        chef, book, etc.);    -   Network Links (e.g., Internet web site where more info        available);    -   Cookbooks where recipe is included;    -   Ingredients and Quantities (including links to ingredient        entries in ingredients data store 370);    -   Nutrition Data (calories, fat, carbohydrates, etc.);    -   Preparation Steps (e.g., chopping, stirring, etc.);    -   Links to Procedures (links to procedure entries in procedures        data store 360);    -   Textual Description of the recipe;    -   Audio Description of the recipe;    -   Video Presentation showing recipe and how prepared;    -   Implements Needed to prepare recipe; and    -   Implement Names (including links to implement entries in        implements data store 380).

Procedures data store 360 includes a number of procedures (e.g., chop,stir, sauté, etc.) used to create various recipes). Procedures datastore 360 includes the following information regarding procedures:

-   -   Name of the procedure;    -   Description—textual description;    -   Purpose—reason(s) why this procedure is used; and    -   Detailed Instructions that may include any or all of:        -   Text—detailed textual instructions;        -   Audio—detailed audio instructions;        -   Video—detailed video instructions; and    -   Links to External Instruction Sources (e.g., links to        instruction sources available on the Internet, etc.)

Ingredients data store 370 includes a number of ingredients used tocreate various recipes. Ingredients data store 370 includes thefollowing information regarding ingredients:

Ingredients Inventory data store 375 includes a list and quantity ofingredients on hand in the user's location (e.g., kitchen, pantry,etc.). Ingredients Inventory data store 375 includes the name of theingredient, a unique identifier used to match the ingredient on hand toits counterpart in ingredients data store 370, a quantity on hand (howmuch of the ingredient is on hand), as well as other metadata about theingredient, such as when the ingredient was last purchased.

-   -   Name—a unique name (identifier) assigned to the ingredient    -   Description of the ingredient that may include any or all of:    -   Text—detailed textual description, including the expiration date        of the ingredient (if applicable), ideal weight of ingredient        (such as a piece of fruit, etc.) ideal texture or appearance of        ingredient;    -   Audio—detailed audio description; and    -   Images—images (photos) of what the ingredient looks like; and    -   Nutrition Data—(calories, fat, carbohydrates, etc.).

Implements data store 380 includes a number of implements used to createvarious recipes. Implements data store 380 includes the followinginformation regarding implements:

-   -   Name—a unique name (identifier) assigned to the implement;    -   Description of the implement that may include any or all of:    -   Text—detailed textual description;    -   Audio—detailed audio description; and    -   Images—images (photos) of what the implement looks like;    -   Purpose—text, audio, multimedia descriptions of the purpose, or        reasons, for using the implement;    -   Usage of the implement that may include any or all of:    -   Text—detailed textual usage description;    -   Audio—detailed audio usage description;    -   Video—detailed video usage description; and    -   Usage Instructions—text, audio, multimedia instructions        regarding how the implement is properly used.

Implements Inventory data store 385 includes a list and quantity ofimplements on hand in the user's location (e.g., kitchen, pantry, etc.).Implements Inventory data store 385 includes the name of the implement,a unique identifier used to match the implement on hand to itscounterpart in implements data store 380, a quantity on hand (how muchof the implement is on hand), as well as other metadata about theimplement, such as when the implement was last purchased and where theimplement is located in the user's location (e.g., which cupboard,drawer, etc.).

FIG. 4 is a flowchart showing steps taken to setup and modify a userprofile used in the interactive cooking preparation device. Userprofiles are stored in profiles data store 400. Processing commences at402 whereupon, at step 404, an identifier is received from the user. Theidentifier is unique to a person whose data is stored in profiles datastore 400. The identifier could be a name, if the names are unique, or aunique code or number assigned to an individual. At step 406, theidentifier received from the user is looked for in profiles data store400. A determination is made as to whether the identifier received fromthe user was found in profiles data store 400 (decision 408). If theuser identifier was found in profiles data store 400 then decision 408branches to “yes” branch 410 whereupon a determination is made as towhether the user wishes to update the existing profile for the user asstored in profiles data store 400 (decision 412). If the user does notwish to update the profile, then decision 412 branches to “no” branch414 which branches back to receive the next user identifier from theuser (or exit if the user wishes to exit the processing shown). On theother hand, if the user wishes to update the existing user profile, thendecision 412 branches to “yes” branch 416, whereupon, at step 418, theexisting user profile that corresponds with the received user identifieris retrieved from profiles data store 400. Returning to decision 408, ifthe identifier received from the user is not found in profiles datastore 400, then the identifier is a new identifier and decision 408branches to “no” branch 419 bypassing steps 412 to 418.

At step 420, the user edits and enters preferences and configurationoptions corresponding to the user identifier that was received at step404. The preferences and configuration options entered at step 420include:

-   -   Disabilities—whether the user corresponding to the identifier        has a disability (e.g., blind, deaf, etc.);    -   Health Profile of the user corresponding to the identifier.        Health profile information includes data such as:        -   the person's weight (e.g., Body Mass Index (BMI), etc.);        -   the person's Blood pressure;        -   the person's Cholesterol; and        -   other health factors.    -   Food Allergies—any food borne allergies that the person may        have;    -   Preferences of the person including:        -   Types of Food the person enjoys (e.g., Italian, vegetarian,            spicy, etc.);        -   Recipe Source(s) the person prefers (e.g., television            personalities, cook books, programs, websites, etc.).

At step 422, the preference data entered and/or edited by the user instep 420 is saved in profiles data store 400 and associated with theidentifier that was entered at step 404. The user can enter additionaluser identifiers and enter/edit profile data regarding other individualsuntil the user wishes to stop editing profile information, at whichpoint processing used to setup and modify user profiles ends.

FIG. 5 is a flowchart showing steps taken to select a recipe using theinteractive cooking preparation device. Processing commences at 425whereupon, at step 426, a selection is received from the user of theinteractive cooking preparation device. The selection includes theparticular user that is using the interactive cooking preparationdevice, such as by the user providing the interactive cookingpreparation device with the user's unique identifier, that is thenretrieved from user profiles data store 400 to retrieve characteristicsof the user as previously described as being included in data store 400.In addition, the user identifies consumers that are the individuals forwhom the user is preparing a recipe. Data regarding the consumers (e.g.,customers, family members, diners, patrons, etc.) is retrieved, ifavailable, from profile data store 400. Among other factors, theretrieved data will inform the user if any of the consumers haveparticular food borne allergies or other food preferences or dislikesthat should be taken into account when preparing a recipe. The user canselect multiple consumers and the selected consumer data is stored incurrent consumer memory area 427.

A determination is made as to whether the user wishes to match selectedingredients with available recipes (decision 428). If the user wishes tomatch selected ingredients with available recipes, then decision 428branches to “yes” branch 430 whereupon, at step 432, an ingredientsbased recipe finder routine is performed that retrieves recipes fromrecipes data store 350 based upon ingredients selected by the user. Onthe other hand, if the user does not want to use selected ingredients ina recipe, then decision 428 branches to “no” branch 434 bypassing step432.

A determination is made as to whether the user wishes to use anyparticular recipe content, such as a favorite cookbook or chef, inselecting a recipe (decision 436). If the user wishes to use particularcontent in selecting a recipe, then decision 436 branches to “yes”branch 438 whereupon a determination is made as to whether the userwishes to use an teaching program to prepare a recipe (decision 440). Ifthe user wishes to use an interactive teaching program, then decision440 branches to “yes” branch 442 whereupon, at step 444, additionalcriteria is received from the user about teaching programs in which theuser is interested, such as a favorite instructor, delivery format, orthe like. At step 446, recipes data store 350 is searched for recipesthat have interactive teaching that match the user's preferences.

Returning to decision 440, if the user wishes to use recipes fromcertain providers but does not wish to limit recipes to those withinteractive instructional content, then decision 440 branches to “no”branch 448 whereupon, at step 450, the user provides additional searchparameters regarding the source of the recipe, such as favorite chefs,cookbooks, etc. At step 452, the user provides additional recipecriteria, such as consumer preferences, food borne allergies, and thelike, that were retrieved at step 426 for the consumers of the meal. Atstep 454, recipes data store 350 is searched for recipes matching theuser's search criteria both in terms of sources as well as consumerpreferences.

Returning to decision 436, if the user does not wish to limit recipes toany particular providers, then decision 436 branches to “no” branch 456whereupon, at step 458, the user provides recipe criteria, such asconsumer preferences, food borne allergies, and the like, that wereretrieved at step 426 for the consumers of the meal. At step 460,recipes data store 360 is searched for recipes matching the user'ssearch preferences entered at step 458.

After the user has entered whatever search criteria is desired, at step462, the results of searching recipes data store 350 are displayed tothe user. The search results displayed can include photographs of therecipes, nutritional data corresponding to the recipes, preparation timeand ingredients needed to prepare the recipes, and any otherrecipe-related data that is included in recipes data store 350. Inaddition, reviews of the recipes, as provided by professionals and/orother cooks, can also be provided to help the user make a recipeselection. The recipes that match the user's search criteria are storedin recipe matches memory area 463. At step 465, the user selects one ofthe recipes displayed at step 465 and the selected recipe is stored inselected recipe memory area 464. At step 466, the interactive cookingpreparation device is used to assist the user in preparing selectedrecipe 464. Processing thereafter ends at 468.

FIG. 6 is a flowchart showing steps taken to interactively prepare arecipe using the interactive cooking preparation device. Processingcommences at 500 whereupon, at step 502, the user selects a recipe fromrecipes data store 350 using one of various means described herein. Theselected recipe is stored in selected recipe memory area 464. At step504, the ingredients needed to prepare the selected recipe are comparedwith the ingredients currently on-hand as set forth in ingredientsinventory data store 375. A determination is made (decision 506), baseon the comparison, as to whether additional ingredients are needed thatare not at the user's location (kitchen, pantry, etc.). If additionalingredients are needed in order to prepare the recipe, then decision 506branches to “yes” branch 508 whereupon, at step 510, the user isinformed that in order to prepare the selected recipe additionalingredients are needed. A determination is made by the user as towhether to continue preparing the recipe (decision 512). If the userdoes not wish to continue (e.g., the user is unable to obtain the neededingredients and would rather prepare a different recipe, etc.), thendecision 512 branches to “no” branch 514 and processing ends at 516. Onthe other hand, if the user wishes to continue preparing the selectedrecipe, then decision 512 branches to “yes” branch 518 whereupon, atstep 520, the user obtains the needed ingredients and, at step 522,ingredients inventory data store 375 is updated according to thequantity of the ingredient(s) obtained.

When the user has the needed ingredients decision 506 branches to “no”branch 524 whereupon, at step 526, the implements needed to prepare theselected recipe are compared with the implements currently on hand(implements inventory data store 385) at the user's location (kitchen,etc.). A determination is made, based on the comparison, as to whetheradditional implements are needed in order to prepare the selected recipe(decision 528). For example, if the selected recipe is for “cremebrulee” and the user does not have a pastry torch, then the user wouldbe instructed to obtain a torch in order to make the selected recipe. Ifone or more additional implements are needed, then decision 528 branchesto “yes” branch 530 whereupon, at step 532, the user is informed of theneeded implements that are not in the user's inventory. A determinationis made by the user as to whether to continue preparing the selectedrecipe (decision 534). If the user chooses not to continue (e.g., theuser cannot obtain the needed implement, etc.), then decision 534branches to “no” branch 536 whereupon processing ends at 538. On theother hand, if the user chooses to continue (e.g., borrowing theimplement from someone, buying the implement, improvising and using adifferent implement, etc.), then decision 534 branches to “yes” branch540 whereupon, at step 542 the user obtains the needed implement (or andifferent implement with which the user will improvise). At step 544, ifthe user purchased the needed implement, then implements inventory datastore 385 is updated.

When the user has the needed ingredients and implements (decision 528branching to “no” branch 546 or the user obtaining the neededimplements), then, at step 548, an ingredients setup routine is executedto setup the ingredients for use with the interactive cookingpreparation device. Processing thereafter ends at 549.

FIG. 7 is a flowchart showing steps taken to setup ingredients usedduring recipe preparation using the interactive cooking preparationdevice. Processing commences at 550 whereupon, based upon theingredients and implements needed to prepare the selected recipe, atstep 552, a surface of interactive cooking preparation device 310 islogically divided into a number of areas where ingredients andimplements will be placed. At step 554 the first ingredient needed forselected recipe 464 is selected. At step 556, a first area of thesurface of interactive cooking preparation device 310 is highlighted. Inone embodiment, the surface is a horizontal display screen. In thisembodiment, the area is highlighted visually by displaying a highlighticon, such as an image of the selected ingredient in the area where theingredient is to be placed. At step 558, an image or other descriptionof the selected ingredient is retrieved from ingredients data store 370.At step 560, the user is requested to place the selected ingredient onthe highlighted surface area. In one embodiment, the instructions andrequests to the user are presented to the user according to the user'scommunication preference (e.g., visually, audibly, etc.) based on theparticular needs and preferences of the user. At step 562, the userretrieves the selected ingredient (e.g., from a pantry, cupboard, etc.)and places the ingredient on the surface of the interactive cookingpreparation device in the highlighted area. At step 564, the interactivecooking preparation device uses sensors to identify the item (theingredient) that the user placed in the highlighted area. At step 566,the interactive cooking preparation device compares the ingredient thatwas retrieved by the user with the requested ingredient. A determinationis made as to whether the user retrieved the correct ingredient(decision 568). If the user did not retrieve the requested ingredient,then decision 568 branches to “no” branch 569 whereupon, at step 570,the user is notified that the wrong ingredient was retrieved andprocessing loops back to have the user retrieve the requested ingredientand place the requested ingredient on the highlighted area of theinteractive cooking preparation device's surface. For example, if therecipe needs “baking soda” but the user retrieved “baking powder”, thenthe user would be notified that the wrong ingredient was retrieved and,possibly prevent the user from using the wrong ingredient in the recipe.On the other hand, if the correct ingredient was retrieved by the userand placed in the highlighted surface area of the interactive cookingpreparation device, then decision 568 branches to “yes” branch 571whereupon a determination is made as to whether there are moreingredients needed for the recipe (decision 572). If more ingredientsare needed, then decision 572 branches to “yes” branch 573 which loopsback to select the next ingredient from the recipe and highlight anothersurface area of the interactive cooking preparation device where thenext ingredient should be placed by the user. This looping continuesuntil all the ingredients have been retrieved and placed on areas of thehorizontal surface, at which time decision 572 branches to “no” branch574 whereupon, at step 575, the user sets up the implements by placingthem in requested areas.

FIG. 8 is a flowchart showing steps taken to setup implements usedduring recipe preparation using the interactive cooking preparationdevice. Processing commences at 576 whereupon, at step 578, the surfacearea of interactive cooking preparation device 310 is divided into areas(if not already divided) for implement and implement placement. At step580 the first implement needed for selected recipe 464 is selected. Atstep 556, a first area of the surface of interactive cooking preparationdevice 310 is highlighted. In one embodiment, the surface is ahorizontal display screen. In this embodiment, the area is highlightedvisually by displaying a highlight icon, such as an image of theselected implement in the area where the implement is to be placed. Ifingredients have already been placed on the surface, then the first areaused to store the first selected implement is the next area of thesurface. At step 584, an image or other description of the selectedimplement is retrieved from implements data store 380. At step 585, theuser is requested to place the selected implement on the highlightedsurface area. In one embodiment, the instructions and requests to theuser are presented to the user according to the user's communicationpreference (e.g., visually, audibly, etc.) based on the particular needsand preferences of the user. At step 586, the user retrieves theselected implement (e.g., from a drawer, cupboard, etc.) and places theimplement on the surface of the interactive cooking preparation devicein the highlighted area. At step 588, the interactive cookingpreparation device uses sensors to identify the item (the implement)that the user placed in the highlighted area. At step 590, theinteractive cooking preparation device compares the implement that wasretrieved by the user with the requested implement. A determination ismade as to whether the user retrieved the correct implement (decision592). If the user did not retrieve the requested implement, thendecision 592 branches to “no” branch 593 whereupon, at step 594, theuser is notified that the wrong implement was retrieved and processingloops back to have the user retrieve the requested implement and placethe requested implement on the highlighted area of the interactivecooking preparation device's surface. For example, if the recipe needs acolander but the user retrieved a bowl instead, then the user would benotified that the wrong implement was retrieved and, possibly preventthe user from using the wrong implement in the recipe. On the otherhand, if the correct implement was retrieved by the user and placed inthe highlighted surface area of the interactive cooking preparationdevice, then decision 592 branches to “yes” branch 595 whereupon adetermination is made as to whether there are more implements needed forthe recipe (decision 596). If more implements are needed, then decision596 branches to “yes” branch 597 which loops back to select the nextimplement from the recipe and highlight another surface area of theinteractive cooking preparation device where the next implement shouldbe placed by the user. This looping continues until all of theimplements have been retrieved and placed on areas of the horizontalsurface, at which time decision 596 branches to “no” branch 598whereupon, at step 599, the user performs the preparation steps setforth in the recipe.

FIG. 9 is a flowchart of the processing performed to perform the stepsset forth in a recipe. Processing commences at 600 whereupon, at step601, the first preparation step is selected from selected recipe 464(e.g., combine various ingredients in a mixing bowl, etc.). At step 602,the selected step is presented to the user according to the user'spreferences (e.g., visually, using multimedia demonstration clip,audibly, etc.). At step 603, the user is instructed to perform theselected preparation step on the surface (e.g., a work area) ofinteractive cooking preparation device 310. At step 604, the interactivecooking preparation device highlights (e.g., visually by generating avisual highlight at the horizontal display screen) in an area proximateto the ingredients and implements needed to perform the currentpreparation step. At step 605, the interactive cooking preparationdevice identifies the ingredients and implements being used by the user,using sensors included in the interactive cooking preparation device andcompares the ingredients/implements being used to those requested in thepreparation step. A determination is made as to whether the user isusing the correct ingredients and implements (decision 606). If the useris not using the correct ingredients and implements, then decision 606branches to “no” branch 607 whereupon, at step 608, the user is notified(alerted) of the problem and instructed to use the correct ingredientsand implements to perform the preparation step of the selected recipeand processing loops back to highlight the areas on the surface wherethe ingredients and implements are located and re-identify theingredients and implements being used by the user. When the user isusing the correct ingredients and implements to perform the currentpreparation step of the selected recipe, then decision 606 branches to“yes” branch 609 whereupon, at step 610, the user's actions aremonitored by the sensors to make sure that the user's actions fallwithin acceptable performance thresholds. A determination is made as towhether the current preparation step of the selected recipe involvescontrol of an automated kitchen appliance (decision 611). If the currentpreparation step involves controlling a kitchen appliance, then decision611 branches to “yes” branch 612 whereupon, at step 613, the applianceis controlled (e.g., turning on the oven to “bake” and setting thebaking temperature to 350° F. On the other hand, if appliance control isnot needed in the current preparation step, then decision 611 branchesto “no” branch 614 bypassing step 613. A determination is made as towhether there are more preparation steps needed to perform to completethe selected recipe (decision 615). If there are more preparation stepsneeded to complete after the user has completed the current preparationstep, then decision 615 branches to “yes” branch 616 which loops back toselect the next preparation step from selected recipe 464 and processthe next step as described above. This looping continues until allpreparation steps needed to prepare the selected recipe have beenperformed, at which time decision 615 branches to “no” branch 617 andprocessing ends at 618.

FIG. 10 is a flowchart of the steps taken to monitor a user's actionswhen following steps in a recipe. Processing commences at 620 whereupon,at step 621 the first step that the user is to perform during theselected preparation step of selected recipe 464 is selected. At step622, a demonstration of the selection action is retrieved fromprocedures data store 360. Examples of procedures that could beretrieved include chopping, dicing, kneading, etc. At step 623, the useris instructed to perform the selected action and the retrieveddemonstration is presented to the user according to the user'spreferences (e.g., visually, audibly, etc.) on interactive cookingpreparation device 310. At step 624, a timer is set according to theamount of time the action is to be performed (e.g., “knead the dough forten minutes,” etc.). At step 625, video devices included in theinteractive cooking preparation device are used to capture images of theuser performing the action. A determination is made as to whether theaction being performed by the user fall within acceptable actionthresholds (decision 626). If the user's actions do not fall withinacceptable action thresholds, then decision 626 branches to “no” branch627 whereupon, at step 628, the user is prompted to alter the actionbeing performed. Such prompts might include requesting that the user“stir faster,” “chop finer,” etc. On the other hand, if the user'scurrent actions fall within the performance thresholds for the action,then decision 626 branches to “yes” branch 629 bypassing step 628. Atstep 630, the timer is decremented. In one embodiment, the timer is alsoadjusted according to the user's actions. For example, if the user wasinstructed to stir for five minutes, but it is determined that the useris stirring too slowly, then the timer can be adjusted upwards so thatthe user stirs the ingredients for a longer period of time. Adetermination is made as to whether the time is complete for performingthe selected action (decision 631). If the time is not complete, thendecision 632 branches to “no” branch 632 which loops back to continuemonitoring the user's actions and prompt the user to alter theperformance as necessary. This looping continues until the time that theaction is to be performed is complete, whereupon decision 631 branchesto “yes” branch 633. A determination is then made as to whether thereare more actions to be performed by the user (decision 634). If thereare more actions to be performed by the user, then decision 634 branchesto “yes” branch 635 which loops back to select the next action needed toprepare selected recipe 464 and have the user perform the next action asdescribed above. This looping continues until all the actions have beenperformed, at which time decision 634 branches to “no” branch 636 andprocessing returns at 637.

FIG. 11 is a flowchart of the steps taken to deliver instructionmaterial to the user of the interactive cooking preparation device.Processing commences at 640 whereupon, at step 641, the user's deliverypreference is retrieved from user profiles data store 400. At step 642,an instruction identifier that corresponds with the instruction that isto be provided to the user is received. A determination is made as towhether the user's profile indicates that the user has a sightimpairment (decision 643). If the user has a sight impairment, thendecision 643 branches to “yes” branch 644 whereupon, at step 645,enhanced content 653 appropriate for a person with a visual impairmentis received from instruction data store 650. An example of enhancedcontent 653 would be audible instructions that are more detailed thanaudible instructions that accompany a video content file. Anotherexample of enhanced content 653 would be Braille output that would bepresented to the user on a Braille output pad included in theinteractive cooking preparation device. At step 646, the enhancedcontent for a person with a vision impairment that was retrieved at step645 is delivered to the user (e.g., played over speakers, presented on aBraille output pad, etc.). Processing then returns at 647.

Returning to decision 643, if the user does not have a sight impairment,then decision 643 branches to “no” branch 648 whereupon a determinationis made as to whether the user has a hearing impairment (decision 649).If the user has a hearing impairment, then decision 649 branches to“yes” branch 654 whereupon, at step 655 enhanced content 652 appropriatefor a person with a hearing impairment is received from instruction datastore 650. An example of enhanced content 652 would be subtitle text(e.g., Closed Captioning) that would be displayed in conjunction with avideo instruction file. Another example of enhanced content 652 would bea detailed textual description displayed on the horizontal displayscreen of the interactive cooking preparation device. At step 656, theenhanced content for a person with a hearing impairment that wasretrieved at step 655 is delivered to the user (e.g., displayed on thehorizontal video display of the interactive cooking preparation device,etc.). Processing then returns at 657.

Returning to decision 649, if the user does not have a hearingimpairment or a sight impairment, then decision 649 branches to “no”branch 658 whereupon, at step 659 standard audio and video are retrievedfrom standard audio/video content 651 according to the user'spreferences. For example, even though the user is not noted as hearingimpaired, the user may request subtitles on the video so that the usercan listen to music or other sounds rather than listening to audibleinstructions. At step 660, the standard content that was retrieved atstep 659 are delivered to the user. Processing then returns at 661.

FIG. 12 is a diagram of a flexible horizontal display screen withactuators underneath to provide tactile feedback to a user of theinteractive cooking preparation device. Top view 662 of horizontalsurface 663 shows placement of multiple actuators 664. In addition,Braille output area 665 is shown with pins that can be actuated to formBraille characters used to communicate with a sight-impaired user.Horizontal surface is a flexible display panel, such as a FlexibleOrganic Light-Emitting Diode (Flexible OLED or FOLED) display panel.Because of the flexible nature of the horizontal display panel surface,actuators 664 (when activated) press against the underside of flexiblescreen surface 663 and can be felt by a user that is touching the top ofthe horizontal surface.

Side view 665 is a partial cross section of flexible horizontal screensurface 663 and three actuators 664. As shown, when actuators 664 areactuated by supplying a current to the actuators, the actuators moveupwards and press against the underside of flexible horizontal screensurface 663. Groups of actuators can be actuated in order to highlightan area of the surface or direct a user to an area of the surface.

Directional example 666 is a top view showing sets of actuators that areactuated using a timing delay to simulate a “rolling” effect. In theexample shown, the three actuators to the left of the center simulate arolling effect to the right, the five actuators to the right of thecenter simulate a rolling effect to the left, the five actuators abovethe center simulate a downward rolling effect, and the four actuatorsbelow the center simulate an upward rolling effect. When operated inthis manner, a user touching the surface of the flexible horizontalscreen surface will be directed to the center location. Those of skillin the art will appreciate that other directions (e.g., diagonal, etc.),while not shown can also be used in a rolling fashion to direct the userto a particular surface area.

Vibration example 667 is a top view shown a group of actuators that arerapidly actuated in order to produce a vibration effect on a particulararea of the flexible horizontal screen surface. If the user was beingasked to use an implement that had been placed on the area, thenvibrating the area would highlight the area and help the user locatedthe needed implement. This would be particularly useful for a personwith a sight disability that is not able to see visual highlightsdisplayed on the flexible horizontal screen surface.

FIG. 13 is a flowchart of the steps taken to provide enhancedinstructional delivery to users with a visual impairment. Processingcommences at 670 whereupon a determination is made as to whether to playaudio content (decision 671). If, based on the user's profile, the userwishes to receive audio content, then decision 671 branches to “yes”branch 672 whereupon, at step 673, the audio content (e.g.,instructions, etc.) are played on the speakers connected to theinteractive cooking preparation device. On the other hand, if the userdoes not wish to receive audio content, then decision 671 branches to“no” branch 674 bypassing step 673. A determination is made as towhether to provide the user with Braille instructions (decision 675). Ifthe user wishes to receive instructions in Braille, then decision 675branches to “yes” branch 676 whereupon, at step 677, the Brailleinstructions are output in a Braille output area of the interactivecooking preparation device. On the other hand, if the user does not wishto receive Braille instructions, then decision 675 branches to “no”branch 678 bypassing step 677. A determination is made as to whether theuser wishes to have the flexible horizontal screen surface vibrated inorder to highlight items placed on the surface (decision 679). If theuser wishes to have the surface vibrated in order to highlight varioussurface areas, then decision 679 branches to “yes” branch 680 whereupon,at step 681, the actuators underneath the display screen that areproximate to the item (e.g., an ingredient, an implement, etc.) areactuated rapidly to create a vibration effect. On the other hand, if theuser does not wish to have the surface vibrated, then decision 679branches to “no” branch 682 bypassing step 681. A determination is madeas to whether the user wishes to receive directional tactile sensationson the surface of the interactive cooking preparation device (decision683). If directional sensations are desired, then decision 683 branchesto “yes” branch 684 whereupon, at step 685, sets of actuators aretriggered in order to “roll” the actuators towards an item, such as aningredient or implement, that has been placed on the surface of theinteractive cooking preparation device. On the other hand, ifdirectional tactile sensations are not desired, then decision 683branches to “no” branch 686 bypassing step 685. Processing then returnsat 687.

FIG. 14 is a flowchart of the steps taken to provide enhancedinstructional delivery to users with a hearing impairment. Processingcommences at 690 whereupon a determination is made as to whether theuser wishes to receive subtitles (e.g., Closed Captioning, etc.) alongwith video images (decision 691). If the user wishes to receivesubtitles, then decision 691 branches to “yes” branch 692 whereupon, atstep 693, the subtitles that accompany the video are either retrievedfrom a data store or generated from the audio track. At step 694, thesubtitles are displayed to the user along with the video. On the otherhand, if the user does not wish to view subtitles, then decision 691branches to “no” branch 695 whereupon, at step 696, the video isdisplayed to the user without subtitles. A determination is made as towhether the user wishes to receive enhanced highlighting on theinteractive cooking preparation device's horizontal display (decision697). If the user wishes to receive enhanced highlighting, then decision697 branches to “yes” branch 698 whereupon, at step 699, the surfacearea display proximate to the item of interest (e.g., an ingredient orimplement placed on the surface, etc.) is highlighted by displayingimages proximate to the item. In one embodiment, the enhancedhighlighting is more intense (e.g., brighter, pulsating, etc.) thannormal highlighting provided on the horizontal surface area. On theother hand, if enhanced highlighting is not desired, then decision 697branches to “no” branch 700 bypassing step 699. A determination is madeas to whether to vibrate the horizontal display surface proximate to theitem of interest (decision 701). If vibration of the display has beenrequested by the user, then decision 701 branches to “yes” branch 702whereupon, at step 703, actuators underneath the surface in the areawhere the item has been placed are actuated causing the surface area tovibrate. On the other hand, if vibration is not desired by the user,then decision 701 branches to “no” branch 704 bypassing step 703.Processing then returns at 705.

FIG. 15 is a flowchart of the steps taken to identify items usingvarious scanners connected to the interactive cooking preparationdevice. Processing commences at 710 whereupon, at step 712, the hardwareconfiguration of the interactive cooking preparation device is read fromhardware configuration data store 711. The hardware configuration dataindicates what sensors are installed in the interactive cookingpreparation device. One example of an interactive cooking preparationdevice is a modified Microsoft Surface™ device. This device has platformsensors that can detect particular tags when tagged-items are placed onor proximate to the device's surface. In one embodiment, Radio FrequencyIdentification (RFID) tags are placed on items and an RFID receivers arelocated around the interactive cooking preparation device so thatRFID-tagged items are identified by the receivers when the items aremoved to the surface of the interactive cooking preparation device. Adetermination is made as to whether platform or RFID scanners areincluded in the interactive cooking preparation device (decision 713).If such scanners are included in the interactive cooking preparationdevice, then decision 713 branches to “yes” branch 714 whereupon, atstep 715, the scanners are used to attempt to read the item's tag (if atag has been attached to the item). A determination is made as towhether a tag is detected on the item (decision 716). If a tag is found,then decision 716 branches to “yes” branch 717 whereupon, at step 736,item data is retrieved based on looking up the item's identifier usingthe sensors and processing returns to the calling routine at 737. On theother hand, if a tag is not found (or cannot be read), then decision 716branches to “no” branch 718.

If the interactive cooking preparation device does not have a platformor RFID scanner (decision 713 branching to “no” branch 719) or if a tagcould not be read using such scanners (decision 716 branching to “no”branch 718), then a determination is made as to whether a barcode readeris installed in the interactive cooking preparation device (decision720). If a barcode reader is installed in the interactive cookingpreparation device, then decision 720 branches to “yes” branch 721whereupon, at step 722, an attempt is made to read a barcode affixed tothe item. A determination is made as to whether the interactive cookingpreparation device was able to read a barcode from the item (decision723). If a barcode was read, then decision 723 branches to “yes” branch724 whereupon, at step 736 the item data is retrieved based on lookingup the item's identifier using the barcode scanner and processingreturns to the calling routine at 737. On the other hand, if a barcodecould not be read from the item, then decision 723 branches to “no”branch 725.

If the interactive cooking preparation device does not have a barcodescanner (decision 720 branching to “no” branch 726) or if a barcodecould not be read using the barcode scanner (decision 723 branching to“no” branch 725), then a determination is made as to whether video inputdevices are installed in the interactive cooking preparation device(decision 727). If video input devices are installed in the interactivecooking preparation device, then decision 727 branches to “yes” branch728 whereupon, at step 729, an attempt is made to match an image of theitem taken with the video input devices with known images of itemsretrieved from item images data store 733. A determination is made as towhether a matching item was found in item images data store 733(decision 730). If a match was found, then decision 730 branches to“yes” branch 731 whereupon, at step 736 the item data is retrieved basedon looking up the item's identifier using the matching image andprocessing returns to the calling routine at 737. On the other hand, theitem does not match one of the items in item images data store 733, thendecision 730 branches to “no” branch 732.

If the item identifier cannot be detected using a scanner or a videoimage (either decision 727 branching to “no” branch 734 or decision 730branching to “no” branch 732), then, at step 735, the user enters theitem identifier directly to the interactive cooking preparation device(e.g., using a keyboard attached to the interactive cooking preparationdevice). This identifier is then used to retrieve the item's data atstep 736 and processing returns to the calling routine at 737.

FIG. 16 is a flowchart of the steps taken to provide standard deliveryof instructional materials based on user preferences. Processingcommences at 740 whereupon, at step 741, the interactive cookingpreparation device reads the user's delivery preferences stored inprofile data store 400. Based on the user's preferences, a determinationis made as to whether the play video content to the user on a displayscreen attached to the interactive cooking preparation device (decision742). If video content is preferred by the user, then decision 742branches to “yes” branch 743 whereupon a determination is made as towhether the user also prefers subtitles (e.g., Closed Captioning) toaccompany the video playback (decision 744). If subtitles are requested,then decision 744 branches to “yes” branch 745 whereupon at step 746,the subtitles that accompany the video are either retrieved from a datastore or generated from the audio track. At step 747, the subtitles aredisplayed to the user along with the video. On the other hand, if theuser does not wish to view subtitles, then decision 744 branches to “no”branch 748 whereupon, at step 749, the video is displayed to the userwithout subtitles. Returning to decision 742, if the user does not wishto view video content, then decision 742 branches to “no” branch 750bypassing steps 744 to 749.

A determination is made as to whether the user wishes to hear audio(decision 751). If the user only wishes to receive audio, then decision751 branches to “yes” branch 752 whereupon, at step 753, the audioinstructions are played on the speakers. On the other hand, if the userdoes not wish to hear audio instructions, then decision 751 branches to“no” branch 754 bypassing step 753.

A determination is made as to whether to vibrate the horizontal displaysurface proximate to the item of interest (decision 755). If vibrationof the display has been requested by the user, then decision 755branches to “yes” branch 756 whereupon, at step 757, actuatorsunderneath the surface in the area where the item has been placed areactuated causing the surface area to vibrate. On the other hand, ifvibration is not desired by the user, then decision 755 branches to “no”branch 758 bypassing step 757.

A determination is made as to whether the user wishes to receiveenhanced highlighting on the interactive cooking preparation device'shorizontal display (decision 759). If the user wishes to receiveenhanced highlighting, then decision 759 branches to “yes” branch 760whereupon, at step 761, the surface area display proximate to the itemof interest (e.g., an ingredient or implement placed on the surface,etc.) is highlighted by displaying images proximate to the item. In oneembodiment, the enhanced highlighting is more intense (e.g., brighter,pulsating, etc.) than normal highlighting provided on the horizontalsurface area. On the other hand, if enhanced highlighting is notdesired, then decision 759 branches to “no” branch 763 bypassing step761.

After the standard delivery preferences of the user have been processedand content has been delivered to the user in the preferred manner, thenprocessing returns to the calling routine at 764.

FIG. 17 is a flowchart showing the steps taken to retrieve implementusage instructions and provide the instructions to the user of theinteractive cooking preparation device. Processing commences at 800whereupon, at step 801, the routine receives an implement identifier(e.g., identifying a carving knife) and an implement request from theuser. A determination is made as to whether the implement request is forimplement care instructions (decision 802). If the request is forimplement care instructions, then decision 802 branches to “yes” branch803 whereupon, at step 804, implement care instructions are retrievedfrom implements data store 380. For example, with a carving knife, thecare instructions might include how to properly clean the knife and howto properly store the knife. On the other hand, if the request was notfor implement care instructions, then decision 802 branches to “no”branch 805 bypassing step 804.

A determination is made as to whether the implement request is forimplement maintenance instructions (decision 806). If the request is forimplement maintenance instructions, then decision 806 branches to “yes”branch 807 whereupon, at step 808, implement maintenance instructionsare retrieved from implements data store 380. For example, with thecarving knife, the maintenance instructions might include how toproperly sharpen the knife. On the other hand, if the request was notfor implement maintenance instructions, then decision 806 branches to“no” branch 809 bypassing step 808.

A determination is made as to whether the implement request is forimplement usage instructions (decision 810). If the request is forimplement usage instructions, then decision 810 branches to “yes” branch811 whereupon, at step 812, implement usage instructions are retrievedfrom implements data store 380.

For example, with the carving knife, the usage instructions mightinclude how to carve a ham or a turkey. On the other hand, if therequest was not for implement usage instructions, then decision 810branches to “no” branch 813 bypassing step 812.

After the user's implement request has been retrieved, then, at step814, the retrieved instructions are delivered by the interactive cookingpreparation device to the user according to the user's deliverypreferences. Processing then returns to the calling routine at 815.

FIG. 18 is a flowchart showing the steps taken to identify and suggestadditional recipes that complement a recipe selected by the user of theinteractive cooking preparation device. Processing commences at 850whereupon, at step 851, nutritional values of selected recipe 464 areretrieved from recipes data store 350 and analyzed. At step 852, thenutritional values of the selected recipe are compared to thresholdsthat have been set for or by the user. Examples of thresholds mayinclude high fat, high carbohydrates, high sugar, high sodium, etc. Adetermination is made as to whether one or more nutritional thresholdshave been exceeded by the selected recipe (decision 853). If nonutritional thresholds have been exceeded, then decision 853 branches to“no” branch 854 whereupon processing ends at 864.

On the other hand, if one or more nutritional thresholds are exceeded bythe selected recipe, then decision 853 branches to “yes” branch 855whereupon, at step 856, recipes data store 350 is searched for recipescomplementary to the selected recipe but that do not exceed thenutritional thresholds. For example, if the user has a nutritionalthreshold to keep fat content below a certain number of grams and theselected recipe is a “devil's food cake,” the search performed at step856 would return an “angel food cake” as a complementary alternativebecause both recipes are for cakes, and the “angel food cake” has anacceptable fat content. The complementary recipes resulting from thesearch performed at step 856 are stored in complementary recipe datastore 857. At step 858, the complementary recipes are presented to theuser. The presentation can include images of the complementary recipes,nutritional values of the complementary recipes and how the nutritionalvalues compare with selected recipe 464, and preparation detailsregarding the complementary recipes, including the preparation timeneeded to prepare the complementary recipe as compared to the selectedrecipe. At step 859, one or more complementary recipes are recommendedto the user due to the fact that the selected recipe exceeded one ormore nutritional thresholds. A determination is made as to whether theuser selected one of the complementary recipes (decision 860). If theuser selects one of the complementary recipes, then decision 860branches to “yes” branch 861 whereupon, at step 862, the selected recipeis replaced with the selected complementary recipe and interactivepreparation instructions are presented by the interactive cookingpreparation device to the user to assist in the preparation of thecomplementary recipe in accordance with the user's instruction deliverypreferences. On the other hand, if the user does not select one of thecomplementary recipes, then decision 860 branches to “no” branch 863whereupon, at step 864, interactive preparation instructions arepresented by the interactive cooking preparation device to the user toassist in the preparation of the selected recipe in accordance with theuser's instruction delivery preferences. Processing thereafter ends at865.

One of the preferred implementations of the invention is a clientapplication, namely, a set of instructions (program code) or otherfunctional descriptive material in a code module that may, for example,be resident in the random access memory of the computer. Until requiredby the computer, the set of instructions may be stored in anothercomputer memory, for example, in a hard disk drive, or in a removablememory such as an optical disk (for eventual use in a CD ROM) or floppydisk (for eventual use in a floppy disk drive), or downloaded via theInternet or other computer network. Thus, the present invention may beimplemented as a computer program product for use in a computer. Inaddition, although the various methods described are convenientlyimplemented in a general purpose computer selectively activated orreconfigured by software, one of ordinary skill in the art would alsorecognize that such methods may be carried out in hardware, in firmware,or in more specialized apparatus constructed to perform the requiredmethod steps. Functional descriptive material is information thatimparts functionality to a machine. Functional descriptive materialincludes, but is not limited to, computer programs, instructions, rules,facts, definitions of computable functions, objects, and datastructures.

While particular embodiments of the present invention have been shownand described, it will be obvious to those skilled in the art that,based upon the teachings herein, that changes and modifications may bemade without departing from this invention and its broader aspects.Therefore, the appended claims are to encompass within their scope allsuch changes and modifications as are within the true spirit and scopeof this invention. Furthermore, it is to be understood that theinvention is solely defined by the appended claims. It will beunderstood by those with skill in the art that if a specific number ofan introduced claim element is intended, such intent will be explicitlyrecited in the claim, and in the absence of such recitation no suchlimitation is present. For non-limiting example, as an aid tounderstanding, the following appended claims contain usage of theintroductory phrases “at least one” and “one or more” to introduce claimelements. However, the use of such phrases should not be construed toimply that the introduction of a claim element by the indefinitearticles “a” or “an” limits any particular claim containing suchintroduced claim element to inventions containing only one such element,even when the same claim includes the introductory phrases “one or more”or “at least one” and indefinite articles such as “a” or “an”; the sameholds true for the use in the claims of definite articles.

1. A computer-implemented method comprising: retrieving an instructiondelivery preference corresponding to a user with a disability, theinstruction delivery preference received by an interactive cookingpreparation device; selecting, by the user, a recipe from a plurality ofrecipes; retrieving, from a data store, a plurality of preparation stepscorresponding to the selected recipe; and providing the plurality ofpreparation steps to the user using the interactive cooking preparationdevice to provide the preparation steps in an alternative delivery modebased on the retrieved instruction delivery preference.
 2. The method ofclaim 1 wherein the disability is a hearing disability and the methodfurther comprises: retrieving a multimedia content corresponding to oneor more of the preparation steps; retrieving a subtitles text contentthat corresponds to the retrieved multimedia content; and displaying themultimedia content and the subtitles text content on a display deviceincluded in the interactive cooking preparation device.
 3. The method ofclaim 2 further comprising: identifying a plurality of locationscorresponding to a plurality of items used during the preparation steps,wherein the plurality of locations are included in a horizontal displaysurface of the interactive cooking preparation device; selecting one ofthe plurality of items is included in a current preparation step of theplurality of preparation steps; and visually highlighting the locationof the horizontal display surface of the interactive cooking preparationdevice that corresponds to the selected item.
 4. The method of claim 1wherein the disability is a sight disability and the method furthercomprises: retrieving an auditory content corresponding to one or moreof the preparation steps; and playing the auditory content on a speakerthat is connected to the interactive cooking preparation device.
 5. Themethod of claim 1 wherein the disability is a sight disability and themethod further comprises: retrieving one or more Braille instructionsthat correspond to one or more of the preparation steps; andtransmitting the retrieved Braille instructions to a Braille output areaincluded in the interactive cooking preparation device.
 6. The method ofclaim 1 wherein the disability is a sight disability and the methodfurther comprises: identifying a plurality of locations corresponding toa plurality of items used during the preparation steps, wherein theplurality of locations are included in a horizontal surface of theinteractive cooking preparation device; selecting one of the pluralityof items is included in a current preparation step of the plurality ofpreparation steps; and vibrating the location of the horizontal surfaceof the interactive cooking preparation device that corresponds to theselected item using one or more actuators located underneath thehorizontal surface.
 7. The method of claim 1 wherein the disability is asight disability and the method further comprises: identifying aplurality of locations corresponding to a plurality of items used duringthe preparation steps, wherein the plurality of locations are includedin a horizontal surface of the interactive cooking preparation device;selecting one of the plurality of items is included in a currentpreparation step of the plurality of preparation steps, the selecteditem being in a selected location; and triggering one or more sets ofactuators located underneath the horizontal surface and outside of theselected location to generate a tactile sensation of rolling towards theselected location of the horizontal surface in order to provide the usertowards the selected location.
 8. An information handling system that isan interactive cooking preparation device, the information handlingsystem comprising: one or more processors; a memory accessible by atleast one of the processors; a nonvolatile storage device accessible byat least one of the processors; a recipe data store stored on thenonvolatile storage device; and a set of instructions which are loadedinto the memory and executed by at least one of the processors toperform actions of: retrieving an instruction delivery preferencecorresponding to a user with a disability; selecting, by the user, arecipe from a plurality of recipes stored in the recipe data store;retrieving, from a data store stored on the nonvolatile storage device,a plurality of preparation steps corresponding to the selected recipe;and providing the plurality of preparation steps to the user using theinteractive cooking preparation device to provide the preparation stepsin an alternative delivery mode based on the retrieved instructiondelivery preference.
 9. The information handling system of claim 8wherein the disability is a hearing disability and wherein theinstructions executed by at least one of the processors performadditional actions comprising: retrieving a multimedia contentcorresponding to one or more of the preparation steps; retrieving asubtitles text content that corresponds to the retrieved multimediacontent; and displaying the multimedia content and the subtitles textcontent on a display device included in the interactive cookingpreparation device.
 10. The information handling system of claim 9wherein the instructions executed by at least one of the processorsperform additional actions comprising: identifying a plurality oflocations corresponding to a plurality of items used during thepreparation steps, wherein the plurality of locations are included in ahorizontal display surface of the interactive cooking preparationdevice; selecting one of the plurality of items is included in a currentpreparation step of the plurality of preparation steps; and visuallyhighlighting the location of the horizontal display surface of theinteractive cooking preparation device that corresponds to the selecteditem.
 11. The information handling system of claim 8 wherein thedisability is a sight disability and wherein the instructions executedby at least one of the processors perform additional actions comprising:retrieving an auditory content corresponding to one or more of thepreparation steps; and playing the auditory content on a speaker that isconnected to the interactive cooking preparation device.
 12. Theinformation handling system of claim 8 wherein the disability is a sightdisability and wherein the instructions executed by at least one of theprocessors perform additional actions comprising: retrieving one or moreBraille instructions that correspond to one or more of the preparationsteps; and transmitting the retrieved Braille instructions to a Brailleoutput area included in the interactive cooking preparation device. 13.The information handling system of claim 8 wherein the disability is asight disability and wherein the instructions executed by at least oneof the processors perform additional actions comprising: identifying aplurality of locations corresponding to a plurality of items used duringthe preparation steps, wherein the plurality of locations are includedin a horizontal surface of the interactive cooking preparation device;selecting one of the plurality of items is included in a currentpreparation step of the plurality of preparation steps; and vibratingthe location of the horizontal surface of the interactive cookingpreparation device that corresponds to the selected item using one ormore actuators located underneath the horizontal surface.
 14. Theinformation handling system of claim 8 wherein the disability is a sightdisability and wherein the instructions executed by at least one of theprocessors perform additional actions comprising: identifying aplurality of locations corresponding to a plurality of items used duringthe preparation steps, wherein the plurality of locations are includedin a horizontal surface of the interactive cooking preparation device;selecting one of the plurality of items is included in a currentpreparation step of the plurality of preparation steps, the selecteditem being in a selected location; and triggering one or more sets ofactuators located underneath the horizontal surface and outside of theselected location to generate a tactile sensation of rolling towards theselected location of the horizontal surface in order to provide the usertowards the selected location.
 15. A computer program product stored ina computer readable medium, comprising functional descriptive materialthat, when executed by an information handling system, causes theinformation handling system to perform actions that include: retrievingan instruction delivery preference corresponding to a user with adisability, the instruction delivery preference received by aninteractive cooking preparation device; selecting, by the user, a recipefrom a plurality of recipes; retrieving, from a data store, a pluralityof preparation steps corresponding to the selected recipe; and providingthe plurality of preparation steps to the user using the interactivecooking preparation device to provide the preparation steps in analternative delivery mode based on the retrieved instruction deliverypreference.
 16. The computer program product of claim 1 wherein thedisability is a hearing disability and further comprising functionaldescriptive material that causes the information handling system toperform additional actions that include: retrieving a multimedia contentcorresponding to one or more of the preparation steps; retrieving asubtitles text content that corresponds to the retrieved multimediacontent; and displaying the multimedia content and the subtitles textcontent on a display device included in the interactive cookingpreparation device.
 17. The computer program product of claim 2 furthercomprising functional descriptive material that causes the informationhandling system to perform additional actions that include: identifyinga plurality of locations corresponding to a plurality of items usedduring the preparation steps, wherein the plurality of locations areincluded in a horizontal display surface of the interactive cookingpreparation device; selecting one of the plurality of items is includedin a current preparation step of the plurality of preparation steps; andvisually highlighting the location of the horizontal display surface ofthe interactive cooking preparation device that corresponds to theselected item.
 18. The computer program product of claim 1 wherein thedisability is a sight disability and further comprising functionaldescriptive material that causes the information handling system toperform additional actions that include: retrieving an auditory contentcorresponding to one or more of the preparation steps; and playing theauditory content on a speaker that is connected to the interactivecooking preparation device.
 19. The computer program product of claim 1wherein the disability is a sight disability and further comprisingfunctional descriptive material that causes the information handlingsystem to perform additional actions that include: retrieving one ormore Braille instructions that correspond to one or more of thepreparation steps; and transmitting the retrieved Braille instructionsto a Braille output area included in the interactive cooking preparationdevice.
 20. The computer program product of claim 1 wherein thedisability is a sight disability and further comprising functionaldescriptive material that causes the information handling system toperform additional actions that include: identifying a plurality oflocations corresponding to a plurality of items used during thepreparation steps, wherein the plurality of locations are included in ahorizontal surface of the interactive cooking preparation device;selecting one of the plurality of items is included in a currentpreparation step of the plurality of preparation steps; and vibratingthe location of the horizontal surface of the interactive cookingpreparation device that corresponds to the selected item using one ormore actuators located underneath the horizontal surface.